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' Part 1 - THE ABSTRACT 

E‘’*sry instructional effort has some collection of procedures 
whereby students are brought in touch with training activities 
and materials. These arrangements have special effects on stu- 
dent and staff members quite apart from the content of the 
curriculum. The ways in which students are moved through the 
curriculum expresses objectives which the management system 
attains by design or, in most cases, by accident. 

The pages which follow describe a management system for 
Individualized treatment of student Instruction. Cert in ob- 
jectives are set for management from which management activities 
and provisions for data storage and use are inferred. The 
result is a comprehensive flow chart of the Instructional se- 
quence with indications of information sources and utilisation. 
Evaluation procedures for measuring the attainment of management 
objectives are also treated. 
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Part 2 - OBJECTIVES FOR MANAGEMENT OF INDIVIDUALIZED INSTRUCTION 

Innovations in training progranis in higher education are 
usually undertaken from one of several major perspectives. Most 
often* content is changed to meet a growing knowledge base or new 
content la added to existing programs to broaden their applica- 
bility. A good exonple of this type of Innovation la the Research 
Training Program launched by USOE in 1967 • New content ~ In the 
form of quautitative anal)3is, information management* etc. - was 
adaed to traditional statistics courses to formulate the program. 
As is often the case, the RTF was designed to train persons for 
new occupational roles — in that instance, roles which were 
anticipated to exist in educational systems. 

A second base for program Innovation may be found in the 
method of training employed. Where new instructional or exper- 
entlal methods have been designed, programs are put forth 
utilizing such techniques. Generally Innovations of this type 
are limited to Departmental or course level offerings and do not 
affect total training programs. 

Another common base for innovation is the management system 
used for moving clients through programs. Thus, we may alter the 
course and credit structure of a program without changing content 
or method of instruction. It is important to note that although 
most Innovations In this category have been focused on entire 
student populations it Is possible to arrange program management 
change from the perspective of the individual student. 
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Every Innovation - no matter what Its base - creates problems 
for the institution of higher learning. All dimensions of institU' 
tional life may have to be altered to accommodate an innovation 
having a broad base and wide impact. This paper discusses an 
innovation in training researchers, evaluators, and developers in 
education which proposes to develop a program based oii all of the 
above perspectives. It outlines the components of the institution 
which will have to be altered and suggests decision rules whereby 
the new program might operate. These latter are sufficiently 
general so as to make the program of potential utility to a wide 
range of organizations engaged in training persons for roles in 
education research, development, and evaluation. 
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Management Objectives 

In this paper we will be primarily concerned with the manage- 
ment system for the proposed program and with acconmodation of 



alternative methods of instruction. This will insure that the 



resulting design will be of widest use and that it will not be 
limited to specific content. 

The objectives of the program - from the point of view of 
management system design - are similar to those outlined by Cooley 
^d Glaser (1968), pp. 2-3. 

"1. The goals of learning are specified in terms of observ- 
able behavior md the conditions under which this 
behavior is to be exercised. 



2, Diagnosis is made of the initial capabilities with 
which the learner begins a particular course of in- 
struction. The capabilities that are assessed are 
those relevant to the forthcoming instruction. 
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3. Educatlotial alternatives adaptive to the initial profile 
of the student are presented to him. The student selects 
or is assigned one of these alternatives. 

4. Student performance is monitored and continuously 
assessed as the student proceeds to learn, 

5. Instruction proceeds as a function of the relationship 
between measures of student performance, available 
instructional alternatives, and criteria of competence. 

6. As instruction proceeds, data are generated for moni- 
toring and Improving the instructional system." 

To this set of objectives we need to add several which are 
related to the over-all content of the program and to its rela- 
tionship to the field it serves. Our program must acconmodate 
new employment roles and must have internal controls and measures 
which will piomote evaluation and development of the program 
itself. These constraints are reflected in the following objec- 



1. Knowledge and skills available to trainees are modular 
in nature and can be aggregated to meet demands of 
particular employee roles. 

2. Follow-up data on trainee performance on the job is 
gathered and used to revise and update training pro- 
gram content. 

3. Students may enter and leave the progran at any point. 
Inter/leave decisions are controlled by trainee 
characteristics and desired skill/knowledge outcomes. 

4. Certification and/or credit is an optional feature of 
the program. Decision rules are available to Insure 
wide interpretation of program content across receiving 
organizations . 

5. Student performance - both during training and subse- 
quent to training - constitutes the measurement opera- 
tion for program evaluation. It also produces a 
baseline of performance against which instructional 
innovations can be Judged, 



tives 
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These objectives are to be attained by reorganiElng instruc- 
tional and management efforts as presently in force in institutions 
of higher learning. Part of this reorganization is directed to 
open options for training which may lie outside of traditional 
educational institutions. These outcomes are attacked in the 
pages to follow through the generalized objective of a program 
design which is based on the eleven objectives noted above and 
which takes into account patterns of student and resource flow 
which may be viewed as 'givens' for this specific design. 

Systan Flows in Training Organizations 

The starting point for design of a system of instruction 
which will meet the above objectives is the Identification of 
major flows of persons, materials and information which the 
training organlzatior, operates upon. These flows give us a 
general input-output model of training depicted in Figure I, 

FIGUEE I 



General Flow Model for 
Training Organizations 




Outputs 
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Figure I shows the first limitation placed on our analysis. 
The program design discussed in this paper is directed entirely 
toward activities enclosed in the box labelled ’Training Organi- 
zation’. This means that some significant influences on the 
instructional program are being left out of consideration. These 
are decisions made by the Institutional Sub-system (Katz and 
Kahnj 1966). This sub-system determines the over-all goals of 
the training organization and makes alterations to inputs to 
the organization based on social, philosophical and economic 
concerns. In the case of the training program under considera-’ 
tion, the institutional sub-system has already decided to examine 
a new training design. Based on the results of design activity, 
the institutional sub-system may elect to Implement the proposed 
training program. Because this sub— system represents diverse 
societal Influences on training, it may elect to make signifi- 
cant alterations in the objectives of training or to change the 
institutional framework within which training takes place. For 
example, a decision to change the location of training from 
traditional educational organizations to an Industrial setting 
would be representative of the scope of decisions made by the 
institutional sub-system. 

Recognizing the above constraint, we can now proceed to 
identify flows internal to the training organization, sub-systems 
which serve Importait functions in the ovfir-all training effort 
and parameters which describe different classes of each sub- 
system, This la done by first addressing the act of instruction 
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as central to any training organization. 

^ 1 ) The Instructional Sub-system 5 

The technical activities of the school take place in the instruc 
tional sub-system. The resources of the training organization are 
applied to bring about desired changes in the behavior of students. 
The objectives of this section of our analysis is to isolate those 
functions which are essential to the performance of Instruction. 

It is important to note that the instructional sub-system does 
not take a given or constant shape. It may vary from the tradi- 
tional self-contained classroom with heavy use of human inputs to 
direct interaction between students and curricula through the 
mediim of a computer. Indeed, the objectives for design of the 
system proposed in this paper suggest basic revisions in present 
conceptions of instruction organization and management in training 
orgsmlzations. Note that one of the objectives of this design 
project was to permit chmge in instruction practice based on data 
gathered during the process of instruction. This adaptive feature 
of the design can best be accomodated by specifying the parameters 
of the instruction effort and noting effects of different parameter 
conditions on instruction organization and support provisions. 

A partial listing of the parameters of instruction would 
Include i 

a) instruction technology i instruction may take place via 
several media - progr^mned instruction, traditional 
lecture /discussion and computer based instruction are 
all possible modes of Instruction. 

b) staffing ratios, technology employed and student flow 
identifies a professional staffing ratio for instruction. 
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c) student flow: numbers of students per unit time and 

demographic descriptors of students. 

d) physical facilities: configuration of environment 

related to technology and number of students to be 
served. 

In terms of our general input-output diagram, the instruc- 
tional sub-system organizes the following flows: 

FIGURE II 

General Flows for 
Instructional Sub-Systems 




Outputs 




Instructional 

Sub-Systems 




Monitoring : 
reallocation 
of resources 




mateWstls use data 
eq uipmen t^Se— dj t a 
data on objective 
staff perf 

ctiange 





The box labelled ’monitoring’ in the above diagram encloses an 
Important feature of the proposed program. The feedback of data 
concerning the outputs of training assists in alternating the para- 
meters of instruction or in changing the pattern of resource 
allocation (aoery, 1969). The diagram also suggests several 
clusters of activities or functions which must exist in order to 



continue instruction. These clusters may or may not constitute 



sub-systems within the training organization. Only detailed 
analysis will detemlne their efficacy. 

Sub-systems derived from Figure I are analyzed below. The 
functions they serve are: 





a) Materials acquisition, storage and delivery. 

b) Objectives generation, modification and organization. 

c) Staff recruitment, assignment and evaluation 

d) Client delivery and placement , 

e) System monitoring, and evaluation. 



In this analysis we follow the model used above to treat the 
inr' 'uctional sub-system. That- Is, we specify the major flows into 
and out of the sub-system and list the parameters which determine 
the organization of the sub-system. For each parameter, we identify 
the features it must have based on the eleven basic objectives of 
the instructional management system, 

2) Materials Sub-system s 

The Materials Sub-system is concerned with identification, pro- 
curement, storage, delivery, and replacement of items which are used 
in Instruction, Such items as books, training programs, technical 
assists, and disposable materials are organized by this sub-system. 
Some of its parmaeters arei 



a) Inventory Control: In the individualized Instruction 

program, instructional materials must be continuously 
inventoried and provisions made for signals to reorder 
depleted materials. Several models for inventory control 
may be applicable to management system objectives (e.g. , 
Relneck, 1968), 

b) Storage and Delivery: The wide range of materials 

needed to meet our objective of providing instructional 
alternatives to students demMids storage of materials 
coded to the objectives of instruction. Coding systems 
must also identify location of materials and procedures 
for access by students and staff members. 

c) Replenishment emd Maintenmce: A signal for reorder 

from Inventory control should cause an examination of 
new materials or devices which may be available, 
provisions must also be made for maintaining technical 
assists «id foy training students and staff members in 



their use 
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3) Objectives Sub-system i 

This sub-system collects and organizes the content of instruc- 
tion. As may already be apparent from our indications of materials 
parameters, objectives play a central role in Integrating the 
activities of all organizational sub-systems. The file of objec- 
tives serves as a method of organizing Information used in all 
organizational sub-systems and as a basis for generating evaluative 
operations for program monitoring and change. Parameters for the 
Objectives Sub-systems are; 

a) Format For Statement of Objectives; An instructional 
system directed to individual performance of students 
dictates concise behavioral statement of objectives. 

It also demands that measurement operations be speci- 
fied for objective attainment and that appropriate 
instructional materia!’ and methods be specified. 

b) Objective Storage and Delivery; Objectives must be 
addressed and stored to permit Individualization of 
student progress through training. The storage 
design must also pemit identification of links 
among objectives related to program outcomes and to 
hierarchical dependencies . 

These two basic parfimieters of the Objectives Sub-system are 
expanded in Part 3 of this paper. There the central importance 
of the Objectives File to the orgailzation and management of the 
training program is made apparent . 

4) Staff Sub-system ; 

This sub-system deals with recruitment, assignment, and com- 
pensation of personnel employed in instruction. Unlike existing 
staffing arrmgements, this sub-system makes direct links between 
program content and staffing as is reflected in the following 








parameters: 
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a) Staff Recruitment I Flexible to Include part time con- 
tracted services in program preparation as well as 
instruction* Recruitment to cover all orgaiiizational 
baaes where needed services might be obtained, 

b) Staff Competencies I Keyed to program objectives . 
Matching of competency list to objective needs 
determines hiring. 

c) Staff Re-training I Training skills of staff members 
listed for possible use in cross training of other 
staff. Schedules for re-training must also be a part 
of data output from this sub-system. 

d) Staff Compensatloni Standard cost analysis of train- 
ing used to detemine staff payment. Payment to be 
fglated to productivity of staff on quantitative 
and/or qualitative basis. 

These parameters define i along with those of the Materials 
Sub-system, a collection of instructional resources directly keyed 
to the wbjectives of instruction. Thus, all resource inputs to 
instruction are identified and controlled by the pattern of objec- 
tives brought about in student behavior. 

5) Client Sub-system : 

The primary support for instruction is dtii-ived from this sub- 
system. Students are identllj.ed and progrMi goals diagnosed. Upon 
completion of instruction, performance data is smnmarlzed and 
evaluation Infomation stored for follow-up of post-training per- 
fomance. 

a) Client Program Expectations i These desired outcomes 
are derived from a diagnostic procedure administered 
on progrMa entry, ProgrMa objectives are thereby 
Identified for each student, 

b) Client Record! Completion of objective signals entry 
of performance on client record. This record is 
matched against job specifications generated in con- 
sultation with employers to determine hiring of student. 
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c) Client Account: Fiscal accounting is based on objec- 

tives attained. Provisions should be made to allow 
for flexible patterns of billing and for possible 
refunds of monies for completion of objectives. 

6) Evaluation and Monitoring Sub-system i 

Program monitoring is carried out by utilizing the information 
output from all program sub-systems. This same Information is com- 
pared against general program objectives and the objectives of the 
management system to initiate program and/or management modifica- 
tion. For the purposes of this report these two functions have 
been treated separately at greater length. Management system 
evaluation appears in Part 7 of this section. 
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Part 3 - PROGRM OBJECTIVES FILE.* An Information 
Base for Educational Management 

Objectives md tlie Control of Inatructlon 

A large part of research and developmeni: effort in education 
Is currently focused on the design and testing of instructional 
systems. These systems are developed to meet certain instructional 
needs and to realize specific objectives related to processes 
whereby Instruction is carried out (Ammentorp, 1970, Part 1). The 
primary concern of this design activity is that of system manage- 
ment which implies both continuous control as well as adaptability 
of the system. 

In the first place, we recognize that an instructional design 
cannot meet the requirements of 'system’ unless it provides for 
some control fimction. This control activity is the representation 
of the concept of feedback in open systems theory (Habers troch, 
1960) . Information is continuously gathered on dimensions of 
instructional activity and the values of information compared with 
levels expressive of system objectives. Thus, in traditional 
educational systems, we might continuously examine average grad# 
points of students in various programs and not# these progrOTis 
where grade point average is below some pre—determined level. 
Managerial control is then exercised to change some feature of the 
instructional system to bring grade point average up to desired 
levels « 

In new instructional systems indicators must be found which 
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can serve the control function provided by the grade point average 
in traditional educational organizations. This indicator must, 
like the grade point average, be ived from the instructional 
process j more specifically, it must express the movement of the 
student through subject matter or his acquisition of knwrfledge 
and skill. In some new Instructional designs the primary indi- 
cator is one related to the rate at which a stud-ent masters 
instructional goals (Wang, 1968). In others, a measure of 
acceleration is derived from basic student performance rates 
(tJMREL, 1970). In either case, parar iters of the instructional 
system may be modified (controlled) as a result of Information 
gathered on system indicators. 

Many of the new educational designs mentioned above are based 
on a collection of subject matter objectives which students meet 
in the course of their training. These objectives generally 
speclfly some action on the part of the student as well as a 
standard of performance against which student action can be 
evaluated (Mager, 1962, Daley & Morreau, 1970, appended). Thus, 
they have the features of measurement and optimum levels for those 
measurements which system indicators must possess. By tying con- 
trol of instruction to measurement of objective attainment of 
students, we can meet both system needs of momentary control and 
long range adaptation. 

Objectives in Instructional Systems 

As we consider structures needed for the statement and stor- 
age of Instructional objectives, it is appropriate to examine the 
nature of decisions which must.be made in^ystem operation and 
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adaptation. At the highest level of generality, these decisions 
relate to (a) the process of instruction and to patterns of stu- 
dent progress and to (b) system output and the degree of corre- 
spondence between output and environmental conditions. These 
classes of decisions are examined below and Inferences drawn as 
to information needs in objective storage. 

Consider, first, the problem of matching instructional out- 
puts to conditions of employment and/or articulation with other 
organizations existent in the environment of our xusuiuctlonal 
system. Resolution of this matching problem results in program 
creation - in the sense of content selection for training - or in 
program adaptation to meet new conditions. It is essentially a 
process of decomposing environmental opportunities from a level of 
societal role to that of specific (behavioral) training objectives. 

Daley and Morreau (1970, appended) suggest that the process of 
role decomposition be carried out .by preparing a series of objec- 
tives of ever increasing levels of specif icty. As an example to 
Illustrate this process, consider the training of an instructional 
progr^mer. First, the role is stated in terms of the major per- 
formance to be exhibited by the incimabent (trainee) • This 
conceptual objective , in turn, is stated in language expressive 
of the activities of the training progrmi. This educational objec- 
tlve becomes the base for program synthesis. These two steps are 
showi dlagrammatically in Figure I. 
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Figure I 

Role Decomposltloni 
Phase I 



Role 



Conceptual 
Object lye 



Educational 
Obj active 



Instructional Design and prepare self- 

Progranmer instructional trair'ng 

programs. 



Training in task 
analysis* frame 
writing and behavior 
modification. 



In effect* the social role has branched out into a tree which 
has as its end points the educational objective or targets which 
training must meet. 

Figure II 

Role Decomposition: 

Phase I As A Tree 

Conceptual Educational 

Role Objective Objective 




The educational objective derived from the above analysis is 
further decomposed into Instructional and behavioral objectives. 
These two categories of objectives represent an extension of the 
tree of interdependencies to finer levels of detail. The instruc- 
tional objective refers to the terminal demonstration of a coherent 
cluster of knowledge and skills to be derived from a component or 
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module of the training program (Silverman, 1966). That terminal 
demonstration Is, in turn, linked to specific bahavloral objec- 
t Ives the student must meet in the course of his training. 

To continue our programing example, we might find the 
following instructional and behavioral objectives J 

Figure III 

Role Decomposition I 
Phase II 



Given a sequence of hu- 
nian behavior, trainee 
performs a task malysls 
including specification 
of stimuli, responses 
and consequence and iden- 
tifies Ifcamlng problem(s). 




Given a video sample of human 
behavior, trainee Identifies all 
action sequences (operants). 

Given a video sanple of human 
behavior, trainee identifies 
consequences (reinforcers) which 
have apparent control over action. 

Given Information derived from 
the above analysis, trainee 
identifies learning problem 
type exhibited in sample be- 
havior (e.g. discrimination). 



Thus, we have carried our training objective from the very gen- 
eral to these specific behaviors which will have to be brought up to 
strength in the trainee. The decisions which follow from information 
concerning our final collection of behavioral objectives are essen- 
tially related to inclusion. We must decide whether a particular 
objective should be included in total training offerings and 
secondly, whether a particular objective may be part of the training 
for more th«i one role. These decision characteristics set certain 
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1) The aggregation of objectives into roles 
must permit the inclusion of any given 
objective in more thMi one role. 

2) Failure to meet any given objective on 
the part of a trainee should result in 
restriction of paths to terminal roles. 

3) Provisions must be made for adding and/ 
or deleting objectives from the file 
without destroying potential for role 
decomposition or aggregation of objec- 
tives. 

The second class of system decisions has to do with student 
progress through a training program. These are essentially de- 
cisions of sequence. The behavioral objectives in our file are 
related to one another in one of three ways; a given objective 
either, 1) preceeds another, 2) is coincident with it, or 3) is 
indifferent to it (or independent of it). These interactions 
among objectives derive either from the structure of the subject 
matter of training or from learning facilitation which may be 
exhibited across many trainees in the course of program operation. 

As a part of the sequence decision category, system managers 
must also be concerned with the units of work assigned to trainees. 
Some objectives are likely to require more time and effort on the 
part of students than others. These differences need to be taken 
into account in determining work load for students and in awards 
of credit toward some final degree, if such is available in the 
training program. Constraints on the objectives storage file 
derived from sequence decisions arei 

1) lach objective must have a unique storage 
address and no duplicate objectives are 
permitted. 
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2) The address of a given objective must 
reflect sequential dependencies among 
objectives . 

3) Additions of deletions to the file 
must preserve the above sequence 
features, 

A Format For Statement of Objectives 

As a first step in building a file of behavioral objectives 
for a training program, a standard format is needed for statement 
of objectives to be considered for inclusion in the Instructional 
offerings of the training system. Building upon the above con- 
straints, we derive the following format. 

Figure IV 

A Format For Stating 
Behavioral Objectives 

Address Nimberi Eeflects sequential dependencies morig objectives. 

Conditions: The conditions under which a trainee is to demonstrate 

mastery performance. 

Action: The specific behavior to be exhibited by the trainee. 

Measures; Specification of the measurement operation to be applied 
to trainee behavior. 

Criterion Level: The value of the measurement outcome which 

signifies mastery performance. 

Materials: toown Instructional materials which can be used to 

bring about desired student behavior. 

A collection of objectives stated in the above format permits 

manipulation for the purpose of system decision making. Algerbraic 

operations c^ be performed on certain file information to derive 

sianmary statistics for program operation and adaptation. Possible 

operations of this nature are treated below. 



o 

ERIC 

hiMfiiifftifffiaaa 




... I 



- 20 - 



Dependency Malntalriance System 

The following procedure lends itself quite easily, with the 
aide of a small computer, to the recording of dependencies among 
the Instructional objectives. With an appropriate search routine, 
a network of tree structure can be generated for any given objective 
Indicated as a terminal objective for a trainee. This network 
would Indicate all prerequisite objectives and permit the drawing 
of a PERT-type network which would indicate the particular sequential 
paths a trainee would need to follow in order to reach the terminal 
objective. This could be easily repeated for any given number of 
terminal objectives. The system also lends itself to easy maintain— 
ance caused by the deletion or addition of objectives to the file, 
or changes in one of the instructional modules which would result 
in differing dependencies with other objectives in the file. Finally, 
if these dependency networks are analyzed in detail every time a change 
occurs, the task is quite formidable and time consuming. This system 
permits changes to be made by the simple binary comparison of one 
objective with other objectives to answer the question, "Is one 
dependent on the other"? 

The basic information for this dependency maintainance (and 
generation) system would be in the form of a matrix constructed as 
follows. Given that there are distinct instructional objectives 
(instructional modules), an "n x n" matric would be created. The binary 
decision concerning each possible pair of objectives would be repre- 
sented in the matrix by having one in the elements of a particular 
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row (corresponding to the Instructional objectives) upon which it 
is dependent, i,e., which must be achieved before the objective being 
considered. All other elements would be zero, Indicating that the 
associated modules are not a prerequisite. Initially, this would 
require N(N - l)/2 binary comr orisons to be made. For a large number 
of objectives, this would be a rather formidable task but could be 
simplified by grouping the objectives by content or some other 
technique, which would make unnecessary a large number of binary 
comparisons. For example. If a set of objectives could be classi- 
fied into two groups dealing with subject matter A and subject matter 
B, and these two subject matters are' logically Independent (such 
as spelling and arithmetic), then none of the objectives in one set 
need be compared with any of the objectives in the main set. Since 
this matrix does not require that the rows and corresponding columns 
be in any particular order, such arrangements could easily be 
facilitated. Thus, if we read across a row of this matrix (corres- 
ponding to an objective) a one would appear under (illustration) the 
corresponding column of every objective which is prerequisite to the 
terminal objective. If we read down a colusui for a given objective, 
there will be a one in the corresponding row for each objective for 
which is prerequisite. In order to add or delete an objective 
we simply add a row or coluiui or delete a row or coluim. If changes 
are made in an Instructional module which make changes in the de- 
pendencies, we need only make changes in one row and column. 
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To proceed with this system, we would then square this matrix. 
If we then scan across a row of the squared matrix, we will find 
that for every inmiediately prerequisite objective, the value one 
in the original iTiatrix has changed to zero in the squared matrix. 

For those objectives which are still prerequisite but not 
immediately prerequisite, a number of equal to or greater than 
one will remain under the appropriate colum. Both of these 
matrixes need not be retained, for example, on tape, disc, or 
computer memory, since they are redundant. If we scan the 
corresponding rows for the original matrix and the squared matrix, 
and identify those elements where ones change to zero, and replace 
the corresponding zero in the squared matrix with a minuc one, 
all of the information has been contained in the squared matrix. 

The generation of dependencies for any particular student and for 
any particular terminal objective, would need only this altered 
squared matrix. We would need to return to the original matrix 
only if changes, deletions, or additions are made in the objective 
file. Numerous procedures are available for identifying recursive 
relationships accidently introduced into the system. 

The following simple example, based on ten objectives, indicates 
how this might work. This could easily be expanded to a large 
number of instructional objectives (instructional modules) since 
the matrix squaring would appear only at the initial creations 
of the system or when changes must be m ade. The squaring of even 
large matrixes would proceed rapidly, with the proper mathematical 
algorithm, since only integer values of one and zero are involved. 
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The PIRT diagram represents the ''eper dencies among our ten 
sample ohjectlves . The student would proceed from right to left, 
objectives one and seven being the "most” terminal of the set of 
ten. This information would be stored in the original matrix, 
identified as Matrix A. If we look at the PERT diagram, we see 
that objectives 2, 3, 4, 5, 6, and 10 are all prerequisite for 
objective one. Therefore In row one of Matrix A, ones appear in 
column 2, 3, 4, 5, 6, and 10, with zeros in the other elements. 
Simllarily, we notice that objectives 6, 9, and 10 are prerequisite 
for objective 8. Therefore in Matrix A, row 8, we find ones in 
column 6, 9, and 10, and zeros in the other columnfs. If we square 
Matrix A and substitute a minus one for those elements which 
change from one to zero, we have Matrix B. This matrix contains 
all of the information necessary for generating prerequisite 
"trees." 

For example, suppose that objective one is selected as a 
terminal objective. If we scan across row one of Matrix B, we 
Instantly see that objectives 2 and 3 are Inunediately prerequisite 
and that objectives 4, 5, and 10 are prerequisite, but at least 
one step removed. We would, therefore, scan across row 2 to see 
which objectives are prerequisite for it, and find that there are 
none, since all of the elements in that row are zero. Next, we 
scan across row 3, and find that objectives 4, 5, and 6 are 
inmiedlately prerequisite for objective 3, Next, we would scan 
across row 4 and find that there are no Inmedlately prerequisite 
isbjactivea for objective 4. Next, we would scan across rows 5 and 
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6, finding in both cases that 10 is an liiffliediataly prerequisite 
objective for both of these. It is obvious, of course, that any 
objectives may be specified as a terminal objective for a given 
trainee, not just the ones to the left most side of the dependency 
diagram. 

The Instructional management system may be consulted to see 
how this would be applied to an actual student. Basically, a 
student would be tested or decisions otherwise made as to what 
point on the branches, moving from right to left, he should begin 
Instruction, based on prerequisite skills and knowledges. 
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Part 5 - FLOW CHART DOCUMENTATION * 



The pages which follow provide explanation and support for 
the significant features of the program flow charts. Each page 
describes a single flow chart symbol according to the format 
below. 

Symbol Title (refers directly to the notation found 
in the symbol) (symbol coordinates) 

Program Function 

Those symbols chosen for description are in some 
way essential to the operation of the program. Their 
special features are noted here as are Important inter- 
actions between this program element and others. 

Inform .tion Utilization 

The type of information employed in carrying out 
the function described above is discussed. Special 
note is made of operations which are to be performed 
with/on information. 

Information Source 



If a form or process exists whereby information 
can be gathered to carry out the program function, 
it is noted here. Where special forms or procedures 
must be developed in order to gather such information, 
an appropriate indication is made. 



* Page nimbers noted in Part 5 content refer to the page 
ntmbers on lower right hand corners of flow cha t ■ 
(Part 4) and not to the continuous nimbering of the 
document pages. 
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S TUDENT RECRUITMENT 
(Blj p. 1) 



Program Function 

Provides basic demand for program output. Combina- 
tion of recruitment outcome plus requests for organiza- 
tional diagnosis (El, p. 1) results In measure of program 
priorities for development and implemaitation, Basic 
control over recruitment is lodged in the content of 
training available. That is, persons are recruited only 
for training currently available - Inquiries for training 
which cannot be given are stored for possible future 
program development. (B3, p. 1) 

Information Utilization 

Basic datum in projected demand for program objectives 
(training programs). Demand as function of t im e is needed 
as is indication of the locus of demand, e.g., organiza- 
tion, student, etc, . 

Information Source 

A standard inquiry/response form is needed to obtain 
information. This form must be designed to facilitate 
coding of information for various program files. Data 
collected via this form should include demographic 
information on clients and organizations, actual and 
anticipated roxes as well as direct keying to the classi- 
fication system of the Program Objectives File, 
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COMPILE STUDE NT PREREQUISITE BACKGROUND 
(Cl, p, 1) 



Prograin Function 

Insures entry level skills for program clients. 

Each program path (see discussion of Program Objectives 
File) has basic skillE #iich are prerequisite to training 
objectives in that path. The accumulated prerequisite 
data across all clients identifies deficiencies which 
may require development of new training programs. 

Ir format ion Utilization 

Students nroceed through instruction if they have 
prerequisites for their chosen program path (see Diag- 
nosis, El, p. 1), New training packages are developed 
for defier encies exhibited for large numbers of students. 

Information Source 



A program based on car' ful definition of skills 
cannot rely on certification of prerequisite skills 
other organizations. Thus, transcripts and other 
certification of training completion plus many norma- 
tive tests become unacceptable data sources. Special 
demonstration tests or exercises must be developed 
for sampling whole patterns of prerequisite skills. 
Special intensive tests may follow as related to 
program path of client (11, j. 1), 
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COMPILE STUDENT T RAINING OBJECTIVES 
itl, p. 1) ^ ^ 

Program Function 

Establishes the clusters of skills desired by an 
organization and/ nr the program path (see Program Ob- 
jectives File) to be followed by the student. This is 
essentially a diagnostic routine whereby roles are 
synthesized for students or role training desired by 
an organization J,s analyzed into skills. The completion 
of objectives is not to preclude the design (or redesign) 
of jobs within organizations. 

Information Utilization 



The list of student objectives re^’iiltlng from the 
diagnostic operation is to be coded to the Program 
Objectives Pile. Initial matching of objectives stated 
by student or organization may be carried out by a con- 
tent based search routine such as the IBM Key Word in 
Context procedure (IBM, 1962). Coded student objec- 
tives are then matched to the Progran Objectives File 
to determine whether student ce be trained (B3, p. 1) , 
(B3 p. 4). The objectives res., ,tlng from diagnostic 
services to organizations can be aggregated into new 
roles (see Program Objectives File) and demand for new 
training objectives used to key training materials 
development. 

Information Source 



Two diagnostic routines must be developedi 

a) student diagnosis: using role desired as point 

of entry to diagnosis, specific desired training 
objectives and program paths can be derived. 

b) organizational diagnosis* skills desired by 
organizations become the basis for systhesizing 
program paths and/or defining new jobs within 
the organization, 

A search procedure must be developed to translate 
objectives of students and organizations into the classi- 
fication system of , the Program Objectives Pile, 
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RESOURCE MATERIALS 
(Ai, p. 2) 



Program Function 

The use of many forms of Instructional materials 
and assists demands continuous, in the sense of "on-line", 
information as to materials availability. These materials 
derive their Im portance from their role in instruction - 
which is to cause the desired student response to occur. 

Information Utilization 

The signal that resources are available activates 
subsequent steps in the provision of a gr' en objective 
to a student. This same information is used again 
(Dl, p. 5) for a second control on entry to Instruction. 
The Resource Inventory (Al, p. 7) is kept current for 
instruction control as well as for the purpose of 
m a terials re-order (pp. 19-20) . 

Information Source 

Source is the Resource Inventory (Al, P» 2). This 
is keyed to objectives and is kept current by a form 
of inventory control probably analogous to that used 
in scientific supply stockrooms (Jensvain, at. al., 

1968) . 
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STAFF LIST 
; (Bl, p. 2) 

Program Function 

Provides Inventory of human resources available, 
for Instruction, materials and program production, 
progran research and development and diagnostic services. 

Information Utilization 

Staff skill and availability constitute pi -or holds 
on student assignment to objectives. Time based infor- 
mation needed to key later availability. 

Information Source 

Staff skill /availability document (El, p. 10), 

(A3, p. 10) . 
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OBJECTIVES REQU IRE PREREQUISITE 
(M, p. 3) 



Program Function 

Program objectives may require specific prerequisites 
over and above the general requirements for program entry 
(Cl, p. 1). This program element insures that specific 
skills, some of which may be acquired during tht- training 
sequence of a particular student, as sent: 1 to a given 
objective exist. 

Information Utilization 



Specific prerequisites are the basis for the testing 
operation carried out during diagnosis of the student 
(Cl, p. 1) . The listing of prerequisites - keyed to 
objectives - is a basic list against which the tested 
skills of the student are matched. 

Information Source 



In early stages of program development, objective 
prerequisites are attached to each objective by program 
writers. Later student entry skills are matched to 
objective performance (Bl, p. 7) to determine prere- 
quisitfea emperically. 
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REMEDIAL WORK 
(Al, p. 4) 



Program Function 

Provides for special Instructional services to 
meet student deficiencies which are not of sufficient 
frequency to warrant Inclusion In the normal sequence 
of objectives. 

Information Utilization 

Used as determinant of student program path (see 
Program Objective File) . Available remedies for pre- 
requisite skill deficiency can be offered to keep the 
student In a program path. If no remedy is available, 
the student must be re-routed through alternate pro- 
gram paths or his training terminated. High frequency 
of prerequisite failure is a signal to program devel- 
opers of an instructional need, 

InforTOation Source 

Available remedies are stored along with other 
objectives in the Progrm Objectives File, 
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OBJECTIVE AVAILABLE 
(B3, p. 4) 



ProgrML Function 

Matches total list of student desired objectives 
with those currently available. Used as a control 
on student assignment to objectives. 

Information Utilization 



Currently available objectives are metched to the 
31st of student desired objectives resulting from the 
diagnostic procediire (B3, p, 4). Matched objectives 
go to Implementation (Al, p. 5). Objectives not cur- 
rently available are held for later assignment (C3. p. 4). 

Information Source 



Objective availability is determined by administra- 
tive considerations such as the state of program devel- 
opment and program priorities. 
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COLLECT FEl > 
(Bl, p. 4) 
(Bl, p. 5) 



Function 



In the present conception of the tvaining operation, 
student payment is not contingent on the preformance of 
the training organization - that is, training outcomes 
are not guaranteed. Pees are either payable in advance 
of training or arrangements must be made for subsequent 
payment , 



Information Utiligatlon 

Fiscal clearance, either in the form of accounts 
paid or deferred payment approval, (B4, p, 12) is used 
as Tl hold on program asslgrment of the student. 

Information Source 



Student account (B4, p, 12) is keyed to objectives. 
If objactivs to be assigned has been paid, instruction 
can proceed. Procedures for deferred payment are in 
need of development. 
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CLASS SIZE 

(Cl, p. 5) 



Program Function 

One of several constraints on use of training 
resources. This control Insures that those training 
sequences which require minimum enrollments for opera- 
tion will be held until such minimums are met. The 
activation of such a hold for an individual student 
is done through the use of a flag on associated ob- 
jectives in the student-s file (C2, p. 5). 

Information Utilization 

Class size information is hold in storage and, 
upon aggregation of minimum enrollment level, used 
to activate a particular instructional sequence. 

Notice of holds on student progress due to inade- 
quate enrollment is also given to students. 

Information Source 



Class size controls are generated In program devel-? 
opment and are keyed to Instructional objectives (see 
Program Objective File) . 
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T RAINING ORDER 
(m, p. 5) 



Program Function 

Many sequences of objectives in the program paths 
exhibit hierarchical dependenclea (see Program Objec- 
tives File) . If a student cannot be given instruction 
(objective blocked) due to resource/s.taff constraints, 
he cannot continue with other objectives until it is 
known that the blocked objective is not essential to 
his Instructional plan. 



Information regarding the hierarchical dependency 
of objectives is used in searching the student's pro- 
gram plan. ve activates a search of 



can be pursued without completion of the blocked objec- 
tive. 

Information Source 

Training order is Initially determined by program 
designers. As large numbers of students complete pro- 
grams, their records can be analyzed in order to de- 
termine training order emperically. 



Information Utilization 



the student 




possible objectives which 
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OBJECTIVE CREDIT 




Program Function 

s. 

The individualized program demands that the student 
be trained only in skills he does not currently possess. 
This means that he may demonstrate proficiency in any 
objective by some testing procedure, A successful 
performance on such test may result in some form of 
credit being given him by the training organization. 

Information Utilization 

Two classes of Information are used in making the 
credit determinatloni 

a) student performance on some demonstration or 
test of the objective. 

b) a payoff table which Indicates credit 7 ,iven 
by the training organization (or other insti- 
tutions) for demonstrated skill. 

Credit determination is made by direct comparison 
of the student’s performance with the credit payoff 
table (Criteria Met B3, p. 6). 

Information Source 

Performance criteria for credit are the same as 
criteria set for mastery of each objective (Bl, p, 7) 
or can be derived from these criteria. The credit 
payoff table must be developed by the training organi- 
zation and/or other credit granting agencies. 
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PERFORmNCE CRITERIA 
(Bl, p. 7) 

Program Function 

Serves as idicatlon of additional skill increments 
for students. Permanent records of criterion perform- 
ance is used in program evaluation to detemine relation 
ship of student criterion levels to subsequent perform- 
ance, Successful criterion performance by students also 
signal possible reinforcement operations in the instruc- 
tion sequence (see Reinforcement of Student Responses) . 

Information Utilization 



Two classes of information are used in determining 
criterion performance! 

a) the value of the student's performance on the 
measurement of the objective, 

b) the previously specified criterion level estab-^ 
lished for mastery of the objective. 

These values are directly compared and the student 
passed into a performance diagnosis routine (Cl, p, 7) 
for faulure to meet or exceed criterion level. 

Information Source 

The Program Objectives File contains a measurement 
operation for each objective. This file also has cri- 
terion levels specified for each objective. Initially, 
levels are determined by program writers* Later, stu- 
dent performance can be analyzed to determine functional 
relationships between performance levels and subsequent 
training and/or job behavior. 
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EVALUATE PERFORMANCE 
(Cl, p. 7) 

Frogram Func tlon 

The performance of students who do not meet program 
criteria is an Important source of information for 
program development and modification. The goal of 
program liters is a series of programa which insure 
attainment of mastery performance by all students who 
possess prerequisite skills. Therefore, performance 
failures must be examined to determine program weak- 
nesses and possible student motivational variables. 

Information Utilization 

Information regarding student performance keyed to 
specific objectives is used to analyze portions of the 
training sequence jo correct instructional errors. 

Reports on student motivation is used in programs where 
systematic reinforcement of student responses is attempced 

Information Source 

Student performance information is derived direccly 
from the various criterion examinations (Bl, p, 1) , 
Performance is keyed to the instructional sequence 
through the Program Objectives File, Student motiva- 
tional variables are determined by inference from pat- 
terns of effort on objectives (El, p, 1) and by direct 
report of behavior in the training setting. 
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STUDENT FILE 
(A3, p. 7) 



Program Function 

The student file is a pernanent record of performance 
in the training organization. It is the source of jnfor” 
matlon for placement of the trainee and contains the list 
of objectiv’es he has mastered. 

Information Utilization 

The student file may contain the original listing of 
objectives the student plans to meet during training 
(El, p, 1), This listing can then be credited with 
completions and potential role designations (see Program 
Objectives File). Print-out of student file content can 
be forwarded directly to employers as part of the place- 
ment function. 

Information Source 

Information concerning total program for the student 
is derived from the Student Training Objectives (11, p. 1) 
File up-dating takes place upon the completion of an objec 
tive at the criterion level of performance (B2, p, 7), 
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ALTERNATE CHOICE 
(Bl, p. 8) 



Program Function 

Students who fall to meet performance criteria for 
an objective and cannot have that failure remedied may 
have alternate program paths yet available to them (see 
Program Objectives File), At this point in the flow 
of events, the completed objectives of the student are 
searched to determine hierarchical dependencies among 
objectives which he may still meet. These dopendencies, 
in turn, Identify program paths open to the student. 

Information Utilization 



The Student File (A3, p. 7) is examined for completed 
objectives. Completed objectives are compared against 
the Program Objectives File to determine program paths 
open to the student, These options are made known to 
the student and he is recycled to (Bl, p, 6). 

Information Source 

Completed objectives are listed in the Student File 
(B2, p, 7), Program Paths are stored in the Program 
Objectives File, 







REFUND CHECK 
(^1, p. 9) 

? 

/ 

Program Funcflon 

By making a retund of a portion of the. fee paid 
for an instructional sequence available to the student 
on completion, a motivational factor may be introduced 
into instruction. This may be necessary especially if 
the training organization does not make use of other 
reinforcement practices (see Reinforcement of Student 
Responses) . Refund can also be supported from the 
standpoint of program development as completed courses 
or sequences provide important data for program change. 

Information Utilization 

A simple decision rule is needed which signals the 
printing of the reftind check upon completion of an 
objective at the specified criterion level. This fea- 
ture of the program must be integrated with the student 
account to prevent administrative complexity In program 
acccunts (see p, 12) , 

Information Source 



The basic decision rule must be determined by pro- 
gron developers and admlnlstratOi.s. The rule may later 
by refined based on the findings of reinforcement studi 
carried out by program staff. 
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RECRUIT STAFF 

(Cl, prio) 



Program Function 

Human resources are used by the training organiza- 
tion for Instruction, course development (including 
research) and cross-tralnir.g of other staff members. 
Persons may or may not be full-time staff members of 
the organization. Recruitment must take into account 
the priority order of training objectives (Bl, p, 10) 
and the availability of staff members (A3, p. 10), 

Information Utilization 

Training objectives desired by the organization 
determine the time pattern of staff needs. The re- 
cruitment effort must include procedures whereby 
sources of staff may be searched for possible contri- 
butors, It is likely that this activity can be or- 
ganized and controlled by the use of a suitable file 
for recruitment. 

Information Source 

Objectives to be met by recruitment are determined 
by the priority order put on program objectives by 
writers and administrators. Information on possible 
contributors could be collected by a suitable instru- 
ment which prospective staff members could complete. 
These data would be input to the file of oblectives 
which could be taught (El, p, 10). 
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COURSE DEVELOPMENT 
(Al, p. li) 

Program Function 

Aa program objectives are identified and available 
Instructional sequences saarched, it may become necessary 
for new sequences to be developed. These needs are met 
by letainlng staff for the davelopment and research of 
spec if ic progiams or for general worlc in rhis ares of 
activity. 

Information Utilization 

Tha file of objectives which can be taught (I'lj p, 10) 
is conipared against needed course development. The 
matched individuals are coiupared against availability 
infonnation (A3^ p* 10) to result in offers of employment. 

Info rmation Sou rce 

Staff information is derived froia te-achablc objectives 
(Elj p, 10) and from availability information (A3, p, 10). 
Course objectives in need of development are those for 
which no resources are on hand or available (Al, p. 2) , 
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CROS S TRAINING 

Tbi. p. 11, 



Program Function 
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Several of the skills needed by program staff members 
may be taught by other staff present or available. Such 
cross- tra:^ ing may be desirable if a large number of 
progranmed lessons must be written and a person can be 
hired to teach staff members programming skills. 

In fo cmat ion Utilization 

C')mmon training needs can be derived from the Program 
Objectives File. Specific staff inputs can be identified 
in the Objectives Teachable File (El, p. 10), These are 
matched to isolate corss-trainlng needs. 

I nformation Source 

Cross-training potential is derived from Oujectives 
Teachable (FI, p, 10) and from Availability Information 
(A3, p. 10) . Needs for staff training are inferred from 
the Program Objectives File, 




ACCOUNTS R ECEIVABLE (p, *13) 

'jlifire are many applications that can be used in handling 
accounts receivablei basic accounting principles do not change. 
Koweverj the basic records used in accounts receivable are the 
account record, statement, credit listing, and collections 
follow-up. Exact forms for these records are dependent upon 
the nature of the business enterprise. Information for these 
records are taken from invoices showing charges for inventory 
items sold or services rendered. Credit memoranda show allow- 
ances such as refunds. Pages 12—13 show a flow-chart of a 
typical accounts receivable procedure applicable to the present 
management system, 

ACCOUNTS PAYABIJ (p. 14-15) 

Accounts payable applicatlc-ns must provide complete informa- 
tion that reveals purchases from one or groups of vendors. A 
purchase Journal is a general necessity (invoice or voucher) 
register). Pages 14-15 outline a flow-chart of an accounts 
payable procedure which functions on a monthly cycle and #ilch 
applies to the present management system. 

The general procedure Involves the following steps i 

1, Audit of invoices for payment, 

2, Posting of cash discounts if any, 

3, Batch total establishment for control, 

4, Posting to purchase Journals, 

5, Preparation of remittance documents, 

6, Posting to disbursement journals. 

7 • Transmittal of remittance to vendor , 
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GENERAL PAYROLL (p. 16-18) 

Payroll management has both a processing and reporting 
function. Both must be flexible. Pages 16-18 show a flow- 
chart of a typical payroll procedure which would apply to the 
present system. Questions that need to be answered during 
processing include; 

1. Is this the proper record? 

2. Has the maxijnum sccial security deduction been reached? 

3. j^e any additional earnings due? 

4. What deductions need to be made? 

The reporting function generates sunmiatlons of the following 
type which apply to a cost effective management system; 

1. Total earnings disbursement 

2, Total federal taxes 

3. Total state taxes 

4, Total for bonds deducted 

5, Total for social security 

6. Year to date quarter totals 

7, Vacation payments 

8. Sick leave actountlng 

INVENTORY CONTROL (p, 19-20) 

Inventory is defined as an Itemized list of current goods or 
assests. Statistics show that approxitnately one-third to one-half 
of a flrm^s total assets may be Invested in its inventory. A 
small percentage reduction in the level of inventory required may 
therefore release a sizeable amount of capital. 

Inventory control offers the only feasible way in which a 
decentralized Inventory can remain constant and under control. 
Computer control removes much of the guesswork that otherwise 



ERIC 



results 




-69- 






For each item in inventory the computer may be progrannned to 

1, Predict tends of use, 

2, Track present stock levels, 

3, Record use or no use data for projection. 

Statistical evaluation of data has shown that 13 cents of 
each sales dollar is spent in physical movement and storage of 
inventory. If the data relative to inventory Is controlled 
through an adequate management system shipping, handling, and 
storage costs may be reduced. 

The determination of Economic Order Quantity (EOQ) for a 
item may be progranmied to provide cost effective results 
in an area which involves up to one-half of a firms total assets. 
EOQ procedures thus provide for the accumulation of data wiilchi 

1* Predict further demands on resources, 

2, Describe the rate of resource use. 

3. Determine when new orders should be placed. 

Define how much should be ordered In terms of the most 
advantageious quantity for procurement and storage. 

5, Gives Information that detarmlnes if reorder quantities 
should be adjusted. 

Pages 19—20 show a flow-chart of a typical control system for 
resources and econoalc order quantity calculations. 
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Part 6 - REINFC ..CEMENT OF STUDENT RESPONSES: 

A Management Option 

In training prograns designed to prepare individuals for 
specific organizational roles, a guide on the job or practicum 
experience can play an Important part in preparing the trainee 
for the demKids of his role. The practicum provides opportuni- 
ties for synthesis of various aspects of training and for the 
application of newly acquired skills to practical problems. 
Consequently, the practicum is an important t -t-ion to be con- 
sidered in Instructional management systems of the type 
discussed here. 

The practicum can be viewed in one of two ways. It can be 
directly included in plmned Instructional sequences. In this 
use, the practlcimi is treated like other instruction oujectives 
and is subject to the control of the objectives file. The 
practicum can, however, be treated as a reinforcement for stu 
dent performance. Here the practlcina is a consequence of 
attainment of objectives by a student. The student gains access 
to the practicim through instructional management sets and con- 
trols contingencies (Hoimne, 1968). These two uses of the 
practlcimi are discussed below. 

1) The Practicum and Training Synthesis: 

The over-all objective of training programs is to bring 
about responses in the student and to Insure that events (stimuli) 
in the work setting will occasion and reinforce those responses. 
The practicum experience cm be a training opportunity to bring 
specific student behavior imder control of events which surround 

the role he is training to fill. Thus, training objectives can 

o 
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be written for the practiciffii according to the format suggested 
in Part 3 of this paper. 

A dlfTlcult problem In this use of the practiciBn is that of 
the proper sequence of practicum objectives in the total training 
pattern. Obviously, responses to be utilized by the student in 
the practicum must first be developed through training activities. 
It is not clear, however, that there are optlmina collections of 
skills essential to effective practicum experiences. This is a 
question which can only be answered by systematic study of 
learning patterns as related to on the Job perfomance. The 
same can be said for the length of the practicvm experienc 
This has generally been adjusted to employment practices of 

receiving organizations. If we treat the practicimi as a part 

/ 

! 

of the Instructional i sequence, it will he necessary to accommo- 

i ■ ' 

I 

date varying arrangements for the practicim based on student 
movement through objectives, 

2) The Practicim As Reinforcement: 

\ 

Reinforcement practices for establishing student responses 
is a major omission in the mariagement plan discussed in this 
paper. No specific provision has been made to provide relri- 
forcement for newly acquired student behavior. Several options 
are suggested in the management plan - such as direct refund of 
student fees for successful completion (Flowcharts, p, 9) - none 
of these can be easily related to those reinforcers which will 
characterize the employment setting where the student will apply 
skills acquired in the training progr^. 
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As an attsnipt to ttifiet the need for relnforceinent of instruc- 
tion Keller arid his associates (Ferster and Perrott, 1968) have 
used social interaction between student and teacher as reinforce- 
nient for learned behavior. Students who acquire a particular 
set of responses are permitted to evaluate students of lesser 
accomplishment and to interact with instructors with increased 
frequency. In this system, contingencies are set which state 
the level of performance m cessary to arrive at each of the 
available reinforcers. In general, the wider the choice of 
reinforcers, the more likely that each trainee will find con- 
sequences desirable to him and will, accordingly, devote con- 
siderable effort to acquiring the behaviors required in training 
(Premack, 1965). 

Because the student has presumably entered training with 
the goal of a particular job in mind, there is some potential 
reinforcing value inherent in the job setting. By using the 
practicum (or interhshlp) as a reinforcer we permii, the student 
to sample available relnforcers on the job and maximize the prob- 
ability that he will find relnforcers which will come to control 
his responses. He will also become aware of organizational 
reward practices aud Increase the likllhood that his training 
will correspond to available career paths (Galbraith, 1968). 

If the practictm is to be used as a reinforcer for student 
behavior careful attention will have to be given to contingencies 
whereby the practicum is included in the student’s program. That 
portion of the mwagement flowchart (p, .7) related to the practi- 

75 



- 73 - 



cum (internship) will require documentation and decision rules to 
meet this need. At the same time, research must be done on optimiun 
location of the practicum in the training sequence and the potential 
reinforcing value of various experiencea Investigated. 
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Part 7 - SYSTm EVALUATION 

Decisions regarding the form and function of the management 
system are based on the degree to which the system meets its 
design objectives. The eleven objectives stated in Part 1 of 
this paper contain the standards against which the proposed 
management system must be evaluated. Each of the design objec- 
tives are re-stated below along with evaluation procedures 
needed to determine the performance of the management system. 

Objective 1 ~ "The goals of learning are specified in 
terms of observable student behavior 
and the conditions under which this 
behavior is to be exercised," 

This objective states a quality control prcblem for the 
Objectives File, New objectives are not to be admitted to the 
File unless they contain conditions for learner behavior, speci- 
fic behavioral actions to be carried out and measures of those 
actions. Evaluation, therefore, is carried out by establishing 
a procedure for sampling of the content of the Objectives File 
on a periodic basis md comparing the resulting sample of 
objectives against the above standards. Due to the Importance 
of the File in the m^agement system, the presence of any ob- 
jectives not meeting standards in the sample is indication of 
need for chmge in system procedures. 

Objective 2 - "Diagnosis is made of the initial capa- 
bilities with which the learner begins a 
particular course of instruction. The 
capabilities that are assessed are those 
relevant to the forthcoming instruction." 

In the preceding discussion, learner capabilities are 
expanded to include diagnosis of roles which the individual may 
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train for during instruction. Insofar as capabilities of the 
learner are concerned, evaluation must match the diagnostic 
procedure against prerequisites listed for the objectives on 
file. This is a sampling - comparison problem. Samples of 
objectives are chosen from the Pile and their prerequisites 
compared with the range of information gathered In diagnosis. 

Only a relatively small proportion of unmatched diagnostic items 
can be permitted; the magnitude of this proportion can be sta- 
tistically determined as a part of the sampling design. 

The minimum condition for a satisfactory role diagnosis is 
a student program of instruction whose objectives match entry 
requisites for a desired role or roles. The evaluation question 
to be answered, ’Have all objectives on file which apply to 
roles desired by the student been identified in diagnosis?'. 
According to the answer to this question program content may be 
altered or the diagnostic procedure modified. 

Objective 3 - "Educational alternatives adaptive to the 
initial profile of the student are pre- 
sented to him. The student selects or is 
assigned one of these alternatives," 

Evaluation of this objective is an extension of the assess- 
ments made in Objective 2 above. In that case the student was 
presented with the pattern of Instructional objectives which would 
qualify him for specific roles for which he wished to be trained. 
In this objective we wish to expand the student's options to in- 
clude other roles yet unknown to (or unconsidered by) him. The 
mln imim condition for a positive evaluation of Objective 3 is at 
least one sequence of instructional objectives available to him 

which Is nOL ^^art of the program path to his stated role goal. 

¥ 
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By sampling student programs, evaluators can compare alter- 
natives presented him with his stated goals » These alternatives 
should state known roles for which he may train along with the 
seQuence of objectives he should pursue in order to Qualify lor 
those roles. Such presentation should also be made following 
completion of sub-routines or sequences during the period he is 
in training. These alternatives should be listed in descending 
order to indicate choices of objectives which lead to maximum 
numbers of role qualifications upon completion of training. 

Objective 4 - ’’Student performance is monitored and 
continuously ansessed as the student 
proceeds to learn.” 

In the proposed management system, the completion cf a given 
objective by the student is a signal for performance evaluation 
which results in a re-assessment of study options open to him. 
Attainment of the objective is detemined by taking a sample of 
students and analyzing their program path. If the student is 
asked co indicate his interest in new training sequences 
periodically, the objective is being met. 

In addition to feedback of instructional results to students. 
Information must be gathered regarding student performance on 
each instructional objective. This results in stmimary achieve- 
ment data on each objective which can be used to assess effec- 
tiveness of instructional procedures. This aspect of monitoring 
is discussed under Objective 6 below. 

Objective 5 — ’’Instruction proceeds as a function of the 
relationship between measures of student 
performuice, available instructional 
altematives and criteria of competence.” 

erIc 79 
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Thls objective can only be judged in a qualitative manner. 
The practices of those engaged in instruction can be analyzed 
to determine the degree to which Indicators of the above dimen- 
sions of instruction are used in making decisions relating to 
i student progress ^ instructional methods and training content, 

' If instruction is managed largely through the use of computers, 

i the decision tables stored in the computer can be studied to 

: determine the use rf the above indicators. In cases of hianan 

management of instruction, sample decisions would have to be 
i evaluated against the content and intent of Objective 5. 

t . Objective 6 - instruction proceeds, data are gener- 

ated for monitoring and improving the 
I instructional system." 

I 

j The several decisions related to monitoring of the instruc- 

I 

! tlonal system have been discussed above. In adaition to these 

I 

j monitoring or operating decisions, there are several aspects of 

i 

I instruction which must be studied in a research mode in order 

i 

that this objective be attained. Two areas of concern central 
to system operation arei 1) sequence and spacing of instruc- 
tional objectives and 2) articulation of training and practlcum 
or internship experiences. 

In order to attain Objective 6, the management system 
must provide for continued study of training sequences used in 
the program. Empirical tests should be maue of alternate sequence 
patterns as related to student performance both during and sub- 
sequent to training. The system must also analyze instructional 
objectives used from the standpoint of a ’unit of work' for 
students. As discussed in the section treating the Objectives 

ii , 
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File (Part 2) this would Involve study of objectives and student 
performance as a multidimensional scaling problem. 

The practlcum experience should also be studied. It must be 
analyzed as to its effectiveness as a means to attaining instruc- 
tional objectives. If the practiciM is used as reinforcement for 
student performance (see Part 6) various plans for administration 
of the practician experience must be tested against demonstrated 
student learnings. 

Objective 7 - "toowledge and skills available to trainees 
are modular in nature and can be aggreated 
to meet demands of particular employee roles.” 

The patterns of objectives evidenced in student programs con- 
stitute the measurement of this objective. The greater the vari- 
ability in the objectives comprising programs, the greater the 
llkllhood that some form of instructional module is being used. 

To further evaluate this objective, it is necessary to compare 
the programs of students who are in training for identical or 
similar roles. The prograns of these students should show greater 
similarity in objectives included than would be the case for the 
progr^s of students chosen at random from the population of 
trainees . 

The aggregation of objectives (or learning modules) to complete 
role training is facilitated by the structure of the Objectives 

File (Part 2). The File must identify program or role paths open 
to the student. Progr^s of study which make use of identical 
objectives should also make use of identical Instructional modules 
if the above objective is to be realized. 
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Objectlve 8 - "Pollow-up data on trainee performanc4 on the 

job is gathered and used to revise and update 
training program content." 

This objective is closely related to the over-all objectives 
of the training program. Those objectives should state the on-the 
job performmces which are to be realized by trainees. Then, the 
evaluation of this objective is carried out by examining the 
degree to which on the job data is used to revise instructional 
content or practice. 

If the progrMEi Objectives File has rules or procedures for 
adding or changing file content, the evaluation of Objective 8 is 
furthered, A sample of objectives can be drawn and revision 
noted. If revisions are drawn out of on-the-job follow-up data, 
then the objective is being met. 

Objective 9 - "Students may enter and leave the program at 
any point. Enter/leave decisions are con- 
trolled by trainee characteristics and desired 
skill /knowledge outcomes . " 

Evaluation of this objective is a two-part problem. First, 
the determination of flexibility in entering and leaving the 
progran must be made. This is done by exMilning the pattern of 
study of a sample of students. Objective 9 is being met if the 
exhibited patterns show variable entry dates and if students have 
left progrMas to return at a later date. 

Second, the rationale for enter/leave decisions must be 
exafflined. These decisions should be based on the desired objec- 
tives of students as derived from the diagnostic interview given 
students at entry. Prerequisites present at the time of entry 
should permit a student to take advanced objectives in the pro- 
grffli, Similarly, the role or roles for which he wishes to 

8 
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prepare should signal his exit from study. Evaluators must again 
exfflaine the program patterns of a sample of students and match 
program decisions to the data derived from the diagnostic inter- 



view. 



Objective 10 - "Certification and/or credit is an optional 

feature of the program. Decision rules are 
available to insure wide Interpretation of 
program content across receiving organiza- 
tions," 

The presence or absence of the above decision rules can be 
readily ascertained by exmining the various program data files. 

The suitability of the decision rules can, in turn, be determined 
by examining the employment history of a sample of students. If 
students are not being denied employment or credit or certifica- 
tion grounds, the objective is being met. 

Objective 11 - "Student perfomance - both during training 

and subsequent to training - constitutes the 
measurement operation for program evaluation. 

It also produces a baseline of performance 
against which Instructional Innovations can 
be judged." 

This is a complex objective in that it is a statement against 
which the evaluation of program objectives can itself be evaluated. 
It implies that over-all progrMi objectives will be stated in terms 
of student performance. Iffective evaluation will then employ 
data on student action to determine the attairmient of program objec- 
tives. 

The data on student performance is also to be used to assess 
the effectiveness of alternate learning designs. This can be 
measured by exmining decisions related to instruction materials 
and practices and assessing the degree to which these decisions are 



K 




based on student peffomancei It iS| of course, iinplled that 
gains in student performance brought about by instruction 
alternatives will be Judged on a cost-benefit basis. 
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APPENDIX I 

FORMAT FOR WRITING EDUCATIONAL OBJECTIVES 
by Marvin Daley and Lanny Morreai^ 
of the UlffiEL staff 



"If we are to study an educational program systematically 
and intelligently we must first be sure as to the educational 
objectives aimed at." 



— Ralph W. Tyler 

in Basic Principles of 
Curriculum and Instruction 



The objective of this research training program is to develop 
a competent educational researcher. To do this it is necessary to 
identify those classes of behavior which must be present in the 
researcher's repertoire aid to construct a program which insures 
that the candidates will acquire the necessary behavior. Planning 
for this program should provide 1) a sequential development scheme 
for the candidates, 2) a systematic measurement procedure for 
evaluating program effects, 3) the collection of data for continuous 
modification and improvement of the program itself, and 4) procedures 
for disseminating the model and the candidates into the educational 
coamunlty. 

Central to the plan is a precise statement of the program 
objectives — "goals for or desired outcomes of the learning expressed 
in terms of the learner's observable behavior" (Montague and Butts, 
1968), including the specific behavior al/measureable event, the 
conditions tmder which this event will occur, and the criteria by 
which it can be judged (Mager) , 

Vflien an instructional program is planned around precise 
Instructional objectives, the following benefits results 1) curriculim 
materials can be selected to meet specific objectives (Gagne, p. 24), 

,,v. m 
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2) curriculum materials can be modified through the analysij of 
learner responses, 3) the learner’s pi — ement in a learning sequence 
can be based upon his present behaviortil repertoire, 4) learnitig 
experiences can be sequenced on a continuiffli to enable the x .mer to 
emit the desired terminal response in the appropriate real-life 
situations, 5) the instructor can specify the precise conditions 
under which the learner should be reinforced (Gagne; Ingmaa) , 

6) redundant or irrelevant behavior and material can be eliminated 
from the continuum while insuring that no precurrent skills are 
omitted, 7) the learner's progress can be empirically assessed, 

8) training time can be reduced (Mager and McCann, 1961) , 9) lueasures 
for program success can be based on postulated program outcomes 
(Burns, 1968), 10) precise conununlcation of program objectives 
to the learner, the instructor, and other professionals is assured, 
11) a baseline of learner responses can be developed for the 
evaluation of further modifications (Gagne) , and 12) the total 
program can be evaluated by independent observers. 

Since there exist no sequentially arranged behavioral objectives 
for preparing a candidate in this field, it becomes necessary for 
the program planners to generate a sequence of objectives which 
meet the criteria of meesureablllty and conmunlcability . Develop- 
jngo,t of this sequence of instructional objectives can be greatly 
facilitated by: 

1, use of a consistent languagd system for stating 
objectives, 

^ 2. selection of alternate activities and materials 

for each objective, 

3, use of a model computer storage/retrieval system for 
access to these objectiyes, 
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4. development of a design for evaluation of the 
objectives. 

Use of a consistent language systein 

The following behavioral model provides the consistent language 
system necessary for writing Instructional objectives. This model 
requires that the program planner ask how the objectives can be met, 
in what format the objectives will be most useable, and how access 
to the objectives can be made quick and easy. 

A, the statement of a conceptual objective , — the generalized 
goal or outcome of the total program. 

B, the statement of an educational objective , — a delineation 
of the conceptual objective stating general categories of skill/ 
knowledge required to meet that objective, 

C, the statement of an instructional objective , — the specific 
classes of behavior which are included in each educational objective. 
The precision necessary for describing behavior at this level can 

be obtained through the use of Sullivan’s synthesis of basic action 
terms I "identify, name, describe, construct, order and demonstrate 
and the subset of behavioral objective verbs accompanying it," (AAAS) 

D, the statement of the behavioral objective , — the terminal 
outcome of the program as specified by this five-part division: 

1. the individual learner — the specification of the 
learner as differentiated from a class or group of 
learners, (All objectives must be written with the 
assui^tlon that an individual learner will be meeting 
that objective, therefore, the phrase, "the learner" 
can be omitted from the statement Itself.) 

2, action — a specific measureable response which the 
learner will emit. Using the category framework. 

88 
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**- identify, name, describe, construct, order, and 
demonstrate" - the action will specify the precise 
observable behaviors which are necessary for meeting 
Che educational objective* 

3. stimulus conditio n — a delineation of the stimulus 
conditions under which the response ,^111 occur. 

These conditions will relate to three general classes t 
a) the environmental conditions present, e.g,, using 
only a digital computer, b) the history of the learner, 
e.g., after completing module one and two, and c) type 
and quantity of a particular stimulus event, e.g., when 
presented with three problems in factor analysis. 

4. measure — a specific statement of the frequency, 
duration, or quality of the response. The measure- 
ment operation Incorporates one of these units, a 
combination of the units, or a derivative from these 
units . 

5. criteria — the level of accuracy at which the response 
will be emitted. This level must be related to the 
terminal level of competence required for independent 
functioning. Criteria for success must be based on the 
behavior of the successful educational research currently 
in the field. 

Alternate activities and materials 

Each objective should be accompanied by alternate media, activity, 
and envirosmental possibilities (Kapf er) which the instructor or the 
learner can select to meet his individual needs. Each objective 
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should be cast In a systematic format which provides an abstract 
of the key components of the objective IndeKed to the classification 
scheme described in this paper (Fig. 1). 



Figure 1 
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Model computer storage retrieval syatem 

Teachers have difficulty in mmaglng even a limited mmaber 
of specified objectives (Eisner j 1966), let alone the hundreds 
which would be required to prepare a researcher. A computer 
storage/retrieval system appears to be imperative for instructor 
and learner access to the thousands of objectives which will be 
met by the trainee. Several cataloging systems for elementary 
curriculim objectives have been developed over the past three 
years. (UMREL, lOX, Clark County) Comparable cataloging systems 
have also been developed for use at the college level “ (Ohio and 
SWSC) demonstrating that the banking, of objectives in the present 
case is not only possible, but is also practical. An objectives 
bank using the technology and hardware provided by computer 
services will not only provide rapid access to objectives but will 
also serve to eliminate redundancy, provide reporting of student 
progress by objectives met, supply data on the effectiveness of 
specific materials in leading to the desired terminal behavior, 
and enable researchers to investigate the effects of independent 
variables on student behavior. 

Evaluation of the objectives 

An objectives based progrraa provides its own criteria for 
evaluation. Responses to the following sequence of questions 
(based upon the behavioral model) provide the program developer 
with an initial evaluation of the adequacy of his objectives. 
Conceptual Obi active i Does the Conceptual Objective accurately 
state the expected outcomes of the progrOT? Educat i onal Objective 
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Have all classes of behavior necessary for successful educational 
research been included? Have any classes of behaviors non-relevant 
to the successful educational research been included? Instructional 
Objective I Have all behaviors encompassed in the educational 
objectives been included? Behavioral Objective ; Can the behavior 
be emitted by m individual learner? Is the action observable? 

Is the action measurable? Have the specific limitations of stimulus 
conditions for that action been stated? Has a specific quantity, 
quality, or duration for eauh action been specified? Does each be- 
havioral objective lead directly to the meeting of one or more 
educational objectives? Are the suggested Instructional materials 
directly related to the meeting of this objective; Have all pre- 
current skills for meeting the objective been delineated and completed? 
General i Is the language system consistent throughout the objectives? 
Can analysis and revision procedures be prepared consistent with the 
stated objectives? 

The objectives based prograoi also provides information for modi- 
fication based upon this analysis. Failure to realize the conceptual 
objectives of the program can result from unrealistic or incomplete 
writing of instructional objectives or sequences of objectives. 
Inadequate placement of the trainee in the sequence, the selection of 
inappropriate activities, media, or materials, or the use of Inadequate/ 
inconsistent evaluation devices. Since the program is based on objec- 
tives and a terminal product has been specified, any of these defi- 
ciencies CMi be corrected and the objectives re-evaluated. 
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